1. Technical Field of the Invention
The present invention relates to an image synthesis display method, and an apparatus therefor, for synthesizing images obtained by multiple vehicular mounted cameras and for displaying the synthesized images. In particular, the present invention relates to an image synthesis display method and apparatus for a vehicle camera for generating in real time a synthesized image that can easily be viewed on a display screen.
2. Description of the Related Art
When an automobile is being driven backward, it is difficult for the driver to obtain an unobstructed view to the rear, and the angle of vision produces a blind spot. Further, even while a vehicle is moving forward, when another vehicle travelling a parallel course reaches a position whereat it is obscured by a door pillar, that vehicle disappears from the driver's sight. Recently, therefore, vehicular mounted cameras have been used to monitor peripheral locations where at the angle of a driver's field of vision produces blind spots, and images that are obtained by the cameras are displayed on a car navigation screen.
FIG. 9A is a top view of an automobile on which vehicle cameras are mounted, and FIG. 9B is a side view. Five cameras 1 to 5 are mounted on this vehicle: camera 1 explicitly monitors the rear of the vehicle; cameras 2 and 4, mounted at the front and the rear on the vehicle's left side, monitor the oblique left rear of the vehicle; and cameras 3 and 5, mounted at the front and the rear on the vehicle's right side, monitor the oblique right rear of the vehicle. Currently, more than five cameras may be mounted on a vehicle, and in some cases a total of eight cameras are used, including two cameras for providing precise monitoring at the rear and two cameras for monitoring the view to the front.
FIG. 10 is a diagram showing photos representing images obtained by the monitoring cameras 1 to 5 in FIGS. 9A and 9B. These images represent views obtained as an automobile is being backed into a parking lot. Since using a screen to sequentially monitor the images obtained by the monitoring cameras 1 to 5 is difficult for a driver, as is shown in FIG. 11, the images obtained by the cameras are converted to obtain those for which the point of origin is a virtual point located above the automobile. The thus obtained images are then synthesized, and an image is obtained that, when displayed on a screen, enables a driver to monitor at a glance the position of his or her automobile relative to peripheral objects and landscape features.
FIG. 12 is a diagram showing the screen display for an example synthesized image. Since the image presented is the view from the virtual point above the automobile, the driver can identify, on the screen, the periphery of the vehicle that is at the dead angle position. In FIG. 12, the image representing vehicle of the driver is not the synthesis image of the camera images (because it is not viewed by a camera), but an image prepared using illustration data.
However, when images obtained by multiple monitoring cameras mounted on a vehicle are merely synthesized, the resulting synthesized image tends not to be easily seen on the screen of a display device. This is because, frequently, while one specific camera may be positioned so that it obtains a clear and easily identified image, another camera may face the sun, or the view obtained by a camera may be shaded from the sun by an adjacent automobile. Since in many cases vehicle cameras are fixed, so that the directions of focus of these cameras can not be controlled, for each camera the state of recorded objects varies in a time transient manner, depending on the roughness of the road, the direction of vehicular movement and changes in the weather. Accordingly, the luminance levels and color balances of the images obtained by the cameras also vary.
As techniques for synthesizing images obtained by cameras, those disclosed in Japanese Patent Publication No. Hei. 10-187929 and Japanese Patent Publication No. Hei. 11-102430 are well known. According to these techniques, to synthesize multiple images obtained by electronic cameras and to obtain a panoramic image, a matching process is used to obtain overlapping areas between adjacent images, and a correction value for two images is obtained, based on the density values of the pixels in the overlapping area, that is used to correct the luminance and the color balance of the synthesized image and to provide a continuous, smoothly synthesized image.
According to these conventional techniques, the images to be synthesized are static images, and are also clear images that have been obtained through the efforts of the photographer, so that for image synthesis, there are very few images that are at all inappropriate.
However, when images obtained by multiple vehicle cameras are synthesized, as is described above, as time elapses the luminance and the color balance for each camera change, and even images that are inappropriate for image synthesization are fetched. Therefore, if such images are present, it is necessary to permit a driver to use a screen to identify, in real time, the peripheral state of a vehicle that is travelling several tens of kilometers an hour, and the conventional techniques, referred to merely for the synthesis of stationary images, can not be directly applied for the synthesis of images obtained by vehicle cameras.